Method to determine optimal number of http2.0 streams and  connections for better qoe

ABSTRACT

Described embodiments provide for dynamically optimizing the number of application layer streams that may be multiplexed into a single transport layer connection, providing the advantages of application layer multiplexing without incurring unnecessary congestion-based network delays. A device may monitor net bandwidth and packet loss rates for a connection, and may dynamically increase and decrease a number of concurrent application layer streams to balance throughput and congestion avoidance. As congestion increases, the device may reduce concurrent stream limits in order to spawn additional transport layer connections, allowing faster congestion recovery and reduced performance impairment.

FIELD OF THE DISCLOSURE

The present application generally relates to application layermultiplexing and dynamic tuning of a number of concurrent applicationlayer streams.

BACKGROUND OF THE DISCLOSURE

Early versions of the HyperText Transport Protocol (HTTP) allowed onlyone request between computing devices to be outstanding at any one timeon a given transport control protocol (TCP) connection: specifically, aresponse needed to be received for any request, prior to sending asubsequent request. For basic systems and web pages, this was adequate,but as the web grew in complexity with pages having embedded graphics,videos, scripts, interactive content, and other elements, requiring eachrequest to be fulfilled prior to submitting a next request was a causeof significant delays. To address this, most web browsers or otherapplications would open a plurality of transport layer connections so asto transmit a plurality of concurrent HTTP requests. However, thisconsumes a large amount of client and server resources (e.g. TCPsockets, transmission and receive buffer memory, etc.), as well asgreatly increasing overhead and reducing network efficiency. Forexample, if a client device opens ten TCP connections in order totransmit ten concurrent requests, the three-way handshaking procedurerequired for each connection results in thirty back-and-forthsynchronization transactions separate from the requests for the desireddata.

HTTP version 2 addresses these issues by providing application layermultiplexing within a single TCP connection, reducing the amount oftransport layer overhead required for communicating multiple applicationlayer requests and responses and reducing latencies associated with thehandshaking procedure. However, due to TCP congestion avoidancealgorithms such as the slow start algorithm, when these multipleapplication layer streams are multiplexed within the single TCPconnection, the resulting congestion window for the connection growssignificantly slower compared to a net congestion window for acorresponding multiple number of TCP connections each carrying a singleapplication layer stream. In case of packet loss, congestion windowreduction differences can result in a significant performance impact,greatly reducing data throughput between the devices, and negativelyaffecting quality of experience (QoE) for users.

BRIEF SUMMARY OF THE DISCLOSURE

The systems and methods discussed herein provide for dynamicallyoptimizing the number of application layer streams that may bemultiplexed into a single transport layer connection, providing theadvantages of application layer multiplexing without incurringunnecessary congestion-based network delays. A device may monitor netbandwidth and packet loss rates for a connection, and may dynamicallyincrease and decrease a number of concurrent application layer streamsto balance throughput and congestion avoidance. As congestion increases,the device may reduce concurrent stream limits in order to spawnadditional transport layer connections, allowing faster congestionrecovery and reduced performance impairment.

In one aspect, the present disclosure is directed to a method formanaging concurrent application layer streams. The method includesestablishing, by a network interface of a first device, a communicationsession with a second device, the communication session comprising aplurality of concurrent application layer streams. The method alsoincludes determining, by a stream manager of the first device, that anet bandwidth utilization of the communication session is above a firstpredetermined threshold. The method also includes determining, by thestream manager responsive to determining that the net bandwidthutilization of the communication session is above the firstpredetermined threshold, that a packet loss rate of the communicationsession is above a second predetermined threshold. The method alsoincludes transmitting, by the network interface to the second device, afirst request to reduce a number of the plurality of concurrent streams,responsive to the determination that the packet loss rate of thecommunication session is above the second predetermined threshold.

In some implementations, the method includes transmitting the firstrequest to reduce the number of the plurality of concurrent streams bytransmitting a packet comprising a SETTINGS_MAX_CONCURRENT_STREAMSparameter with a predetermined value smaller than the number of theplurality of concurrent streams. In some implementations, the methodincludes subsequently determining, by the stream manager, that thepacket loss rate of the communication session is below the secondpredetermined threshold; and transmitting, by the network interface tothe second device, a second request to increase the number of theplurality of concurrent streams, responsive to the determination thatthe packet loss rate of the communication session is below the secondpredetermined threshold. In a further implementation, the methodincludes subsequently determining, by the stream manager, that the netbandwidth utilization of the communication session is below the firstpredetermined threshold; and responsive to determining that the netbandwidth utilization of the communication session is below the firstpredetermined threshold, transmitting, by the network interface to thesecond device, a second request to reduce the number of the plurality ofconcurrent streams. In a still further implementation, the first requestcomprises an identification of a lower number of concurrent streams thanthe second request.

In some implementations, the communication session comprises a hypertexttransport protocol v2 (HTTP2) session. In some implementations, themethod includes transmitting the first request to reduce the number ofthe plurality of concurrent streams by transmitting a settings framecomprising an identification of one or more configuration parameters forthe communications session.

In another aspect, the present disclosure is directed to a method formanaging concurrent application layer streams. The method includesestablishing, by a network interface of a first device, a communicationsession with a second device, the communication session comprising aplurality of concurrent application layer streams. The method alsoincludes receiving, by the network interface from the second device, afirst request to reduce a number of the plurality of concurrent streams,transmitted responsive to a determination by the second device that (i)a net bandwidth utilization of the communication session is above afirst predetermined threshold, and (ii) a packet loss rate of thecommunication session is above a second predetermined threshold; andtransmitting via a first stream of the plurality of concurrent streams,by the network interface responsive to receipt of the first request toreduce the number of the plurality of concurrent streams, a packetcomprising an indicator that at least one stream of the plurality ofconcurrent application layer streams is closed.

In some implementations, the method includes receiving the first requestto reduce the number of the plurality of concurrent streams by receivinga packet comprising a SETTINGS_MAX_CONCURRENT_STREAMS parameter with apredetermined value smaller than the number of the plurality ofconcurrent streams. In some implementations, the method includesreceiving a second request to increase the number of the plurality ofconcurrent streams, by the first device from the second device,transmitted responsive to a determination by the second device that thepacket loss rate of the communication session is below the secondpredetermined threshold; and transmitting via the first stream of theplurality of concurrent streams, by the network interface responsive toreceipt of the second request to increase the number of the plurality ofconcurrent streams, a second packet comprising the plurality ofconcurrent application layer streams. In a further implementation, themethod includes receiving a second request to reduce the number of theplurality of concurrent streams, by the first device from the seconddevice, transmitted responsive to a determination by the second devicethat the net bandwidth utilization of the communication session is belowthe first predetermined threshold; and transmitting via the first streamof the plurality of concurrent streams, by the network interfaceresponsive to receipt of the second request to reduce the number of theplurality of concurrent streams, a packet comprising an indicator that asecond at least one stream of the plurality of concurrent applicationlayer streams is closed. In a still further implementation, the firstrequest comprises an identification of a lower number of concurrentstreams than the second request. In some implementations, thecommunication session comprises a hypertext transport protocol v2(HTTP2) session. In some implementations, the method includes receivingthe first request to reduce the number of the plurality of concurrentstreams by receiving a settings frame comprising an identification ofone or more configuration parameters for the communications session.

In another aspect, the present disclosure is directed to a system formanaging concurrent application layer streams. The system includes anetwork interface configured to establish a communication session with asecond device, the communication session comprising a plurality ofconcurrent application layer streams; and a stream manager. The streammanager is configured to determine that a net bandwidth utilization ofthe communication session is above a first predetermined threshold, anddetermine, responsive to determining that the net bandwidth utilizationof the communication session is above the first predetermined threshold,that a packet loss rate of the communication session is above a secondpredetermined threshold. The network interface is further configured totransmit, to the second device, a first request to reduce a number ofthe plurality of concurrent streams, responsive to the determinationthat the packet loss rate of the communication session is above thesecond predetermined threshold.

In some implementations, the network interface is further configured totransmit a packet comprising a SETTINGS_MAX_CONCURRENT_STREAMS parameterwith a predetermined value smaller than the number of the plurality ofconcurrent streams. In some implementations, the stream manager isfurther configured to subsequently determine that the packet loss rateof the communication session is below the second predeterminedthreshold; and the network interface is further configured to transmit,to the second device, a second request to increase the number of theplurality of concurrent streams, responsive to the determination thatthe packet loss rate of the communication session is below the secondpredetermined threshold. In a further implementation, the stream manageris further configured to subsequently determine that the net bandwidthutilization of the communication session is below the firstpredetermined threshold; and the network interface is further configuredto transmit, to the second device, a second request to reduce the numberof the plurality of concurrent streams, responsive to determining thatthe net bandwidth utilization of the communication session is below thefirst predetermined threshold. In a still further implementation, thefirst request comprises an identification of a lower number ofconcurrent streams than the second request. In some implementations, thenetwork interface is further configured to transmit a settings framecomprising an identification of one or more configuration parameters forthe communications session.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects, aspects, features, and advantages ofthe present solution will become more apparent and better understood byreferring to the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1A is an illustration of an embodiment of multiplexing applicationlayer connections via a transport layer connection;

FIG. 1B is an illustration of an embodiment of a model of a networkstack;

FIG. 1C is a flow diagram of an embodiment of multiplexing applicationlayer connections via a transport layer connection;

FIG. 2 is a block diagram of an embodiment of a computing device fordynamically optimizing the number of application layer streams that maybe multiplexed into a single transport layer connection; and

FIG. 3 is a flowchart of an embodiment of a method for dynamicallyoptimizing the number of application layer streams that may bemultiplexed into a single transport layer connection.

The features and advantages of the present solution will become moreapparent from the detailed description set forth below when taken inconjunction with the drawings, in which like reference charactersidentify corresponding elements throughout. In the drawings, likereference numbers generally indicate identical, functionally similar,and/or structurally similar elements.

DETAILED DESCRIPTION

Early versions of the HyperText Transport Protocol (HTTP) allowed onlyone request between computing devices to be outstanding at any one timeon a given transport control protocol (TCP) connection: specifically, aresponse needed to be received for any request, prior to sending asubsequent request. For basic systems and web pages, this was adequate,but as the web grew in complexity with pages having embedded graphics,videos, scripts, interactive content, and other elements, requiring eachrequest to be fulfilled prior to submitting a next request was a causeof significant delays. To address this, most web browsers or otherapplications would open a plurality of transport layer connections so asto transmit a plurality of concurrent HTTP requests. However, thisconsumes a large amount of client and server resources (e.g. TCPsockets, transmission and receive buffer memory, etc.), as well asgreatly increasing overhead and reducing network efficiency. Forexample, if a client device opens ten TCP connections in order totransmit ten concurrent requests, the three-way handshaking procedurerequired for each connection results in thirty back-and-forthsynchronization transactions separate from the requests for the desireddata.

HTTP version 2 (HTTP/2, e.g. as described in the proposed InternetEngineering Task Force (IETF) RFC 7540, incorporated by reference hereinin its entirety) addresses these issues by providing application layermultiplexing within a single TCP connection, reducing the amount oftransport layer overhead required for communicating multiple applicationlayer requests and responses and reducing latencies associated with thehandshaking procedure. For example, FIG. 1A is an illustration of anembodiment of multiplexing application layer connections 106A-106N via atransport layer connection 104 between two computing devices 100A, 100B(referred to generally as computing device(s) 100). Computing devices100 may comprise any type and form of computing device, includingdesktop computers, laptop computers, embedded computers, tabletcomputers, wearable computers, smartphones, workstations, rackmountcomputers, networked or smart appliances, set top boxes, video gamesystems, home automation systems, or any other type and form of device.Computing devices 100 may be physical devices, or virtual devicesexecuted by one or more physical devices, and may be deployedindividually, or in a cluster, farm, or cloud.

Computing devices 100 may establish a transport layer connection 104over a network 102. Network 102 may comprise any network or combinationof networks, including local area networks (LANs), metropolitan ormedium area networks (MANs), wide area networks (WANs) such as theInternet, cellular networks, satellite networks, virtual privatenetworks, mesh networks, peer to peer networks, or any other type andform of network. Network 102 may be homogenous or heterogeneous, and maybe carried via any type of physical medium, including Ethernet, coaxialcable, fiber optic, cellular data, microwave, RF, optical over air, orany other type of medium. Multiple devices may be deployed asintermediaries between computing devices 100, including gateways,switches, routers, hubs, firewalls, network accelerators, or otherdevices.

Transport layer connection 104 may comprise any type and form oftransport layer connection, such as a transport control protocol (TCP)or user datagram protocol (UDP) connection. Referring briefly to FIG.1B, illustrated is an embodiment of a model of a network stack 120 basedon the open systems interconnect (OSI) model. A packet may comprise apayload at a higher level, e.g. the application layer, encapsulated inheaders (and trailers or footers, in some implementations) of lowerlayers. Thus, for example, an application layer HTTP packet may becarried as a payload of a transport layer packet (which, itself, may bea payload of a network layer packet; which may be a payload of a datalink layer frame; which may be transmitted via a physical layerprotocol).

Returning to FIG. 1A, in application layer multiplexing protocols suchas HTTP/2, a transport layer connection 104 may carry a plurality ofapplication layer connections 106A-106N (referred to generally asapplication layer connection(s) 106, HTTP connection(s) 106, applicationlayer stream(s) 106, or by similar terms). Headers and payloads ofapplication layer connections 106 may be multiplexed or concatenatedtogether within the payload of the transport layer connection 104,frequently with interstitial headers or stream identifiers to help therecipient device demultiplex the streams, which may be of varyinglengths. This may reduce delays associated with connection establishmentand reduce transport layer overhead compared to individuallyestablishing transport layer connections for each application layerstream.

For example, FIG. 1C is a flow diagram of an embodiment of multiplexingapplication layer connections via a transport layer connection. Thetransport layer connection may be established via a three-way handshakeprocedure 140, in some implementations, as in the TCP protocol, with asynchronization (SYN) packet, an acknowledgement to the SYN packetestablishing mutual configuration settings (SYN ACK), and a furtheracknowledgment of the SYN ACK packet (ACK).

In many implementations, a first device may then transmit an applicationlayer request 142, such as a GET request for a web page (e.g.GET/index.html). The second device may respond with the page, and may insome implementations, include a header specifying that multipleapplication requests may be multiplexed together in a single request. Insome implementations, the header may include an identifier of a maximumnumber of application layer requests that may be multiplexed, such as aSETTINGS_MAX_CONCURRENT_STREAMS parameter or similar parameter. This maybe sent in the same packet as the response to the request, or in aseparate packet (e.g. a settings packet).

The web page may include references to other documents and data, such asembedded images (e.g. “image.jpg”), scripts (e.g. “script.js”), orcascading style sheets (e.g. “styles.css”) that are required for fullyrendering the page. The first computing device may parse the page toidentify these references and may transmit the requests, multiplexed inaccordance with the maximum number of concurrent streams parameter, viathe single transport layer connection 144. The second device maysimilarly respond with the documents and data, similarly multiplexed, asshown. The responses may be transmitted in the same order as therequests, or in different orders. Accordingly, the first computingdevice may transmit multiple requests without needing to wait for eachrequest to be fulfilled, greatly reducing time to load the page, andwithout performing additional handshakes as required when establishingseparate transport layer connections for each application layer request.

Although shown as a communication between two devices, in manyimplementations, the first computing device 100A may establishcommunications with multiple second computing devices. For example,after receiving the web page, the first computing device 100A mayrequest embedded images from a first data server, scripts from a seconddata server, etc. Any number of requests to any particular server may bemultiplexed together within one transport layer connection.

While multiplexing application layer requests into a transport layerconnection may reduce connection establishment overhead and thusincrease data transmission efficiency, due to TCP congestion avoidancealgorithms such as the slow start algorithm, the congestion window forthe single connection grows significantly slower compared to a netcongestion window for a corresponding multiple number of TCP connectionseach carrying a single application layer stream. For example, at thetransport layer, a congestion window may be increased according to aslow-start algorithm at a predetermined rate, regardless of whether thatconnection carries a single HTTP request or response or an HTTP/2 streamencapsulating a plurality of requests or responses. In the slow-startalgorithm, a congestion window is increased by one for eachacknowledgement (ACK) received, effectively doubling the window size foreach request-response round trip. A first pair of devices communicating10 multiplexed requests via a single HTTP/2 over TCP connection willtherefore have a congestion window that increases in size from n to 2nto 4n to 8n, etc. However, a second pair of devices communicating viaten TCP connections, each with a single HTTP request, will have similarcongestion windows per connection, and therefore a total congestionwindow of 10n, 20n, 40n, 80n, etc., resulting in potentially much fasterincrease in total bandwidth.

Furthermore, in case of packet loss, congestion window reductiondifferences can result in a significant performance impact, greatlyreducing data throughput between the devices, and negatively affectingquality of experience (QoE) for users. For example, in someimplementations, the maximum window size of the single connectionbetween the first pair of devices may be divided amongst the multipleconnections of the second pair of devices discussed above. Assuming thatthese connections have been stable for some time, the congestion windowof the single TCP connection for the first pair of devices may be 100packets; and the congestion windows for each of the ten TCP connectionsfor the second pair of devices may be 10 packets, resulting in equal netbandwidth per round trip time for each pair of devices. Given a TCPalgorithm that reduces window size by half in response to a packet drop,if the single TCP connection between the first pair of devices drops apacket, the window is reduced to 50 packets. However, if a packet isdropped on one of the ten TCP connections between the second pair ofdevices, that one connection's congestion window is reduced to 5packets, but the other nine connections' windows remain at 10 packets,resulting in a total window size of 95 packets. Worse, if a secondpacket is dropped, the single TCP connection between the first pair ofdevices may reduce its window to 25 packets. Conversely, if the seconddropped packet is on the same one of the ten TCP connections between thesecond pair of devices, its window is reduced to 2 packets (inimplementations that round down to an integer number of packets), whilethe other nine connections remain unmodified, resulting in a totalwindow size of 92 packets. If instead, the second dropped packet is on adifferent connection, then the window size may be reduced to 90 packets(or possibly remain at 95 packets, if, for example, the first connectionhas recovered and re-increased its window size to 10 packets).

Thus, it may be important to dynamically balance the competing pressuresof reduced overhead and connection efficiency of multiplexed applicationlayer communications, and increased window size growth and congestionresiliency of multiple transport layer connections. The systems andmethods discussed herein provide for dynamically optimizing the numberof application layer streams that may be multiplexed into a singletransport layer connection, providing the advantages of applicationlayer multiplexing without incurring unnecessary congestion-basednetwork delays. A device may monitor net bandwidth and packet loss ratesfor a connection, and may dynamically increase and decrease a number ofconcurrent application layer streams to balance throughput andcongestion avoidance. As congestion increases, the device may reduceconcurrent stream limits in order to spawn additional transport layerconnections, allowing faster congestion recovery.

Specifically, in some implementations, when there is a large amount ofbandwidth available, the system may utilize fewer transport layerconnections with more application layer streams multiplexed in eachconnection. Similarly, as latency increases, the system may utilizefewer transport layer connections with more application layer streamsper connection. This may increase efficiency by reducing overhead.However, as packet loss rates increase, the system may use moretransport layer connections and fewer application layer connections.

In one implementation, the device may modify a concurrent stream valuein a configuration packet or header of a packet, such as theSETTINGS_MAX_CONCURRENT_STREAMS value in the HTTP/2 protocol. In someimplementations, the concurrent stream value may be set to a valueproportional to the number of packet drops or losses from a previousconnection, or may be set to a default value. The device may measure arate of packet drops or losses and dynamically tune the concurrentstream value to limit the number of streams, forcing the other device itis communicating with to spawn additional transport layer connections tocarry additional application layer communications or streams. Forexample, if the connection experiences a large number of packet drops,the device may set a SETTINGS_MAX_CONCURRENT_STREAMS to a small number(e.g. 2 or 3), and causing other devices to create more transport layerconnections.

Thus, in one example implementation, given a bandwidth threshold B′ anda packet loss threshold P′ (e.g. 10 packets per second, or any othersuch value):

IF current bandwidth B > B′ { IF current packet loss rate P > P′ { SetSETTINGS_MAX_CONCURRENT_STREAMS to N1 for that connection; } ELSE { SetSETTINGS_MAX_CONCURRENT_STREAMS to N2 } ELSE { SetSETTINGS_MAX_CONCURRENT_STREAMS to N3 };with N1<N3<N2. For example, in one such implementation, N1 may be equalto 2, N2 may be equal to 20, and N3 may be equal to 10.

Accordingly, the system can balance multiplexing for better efficiencywith optimized congestion avoidance, resulting in lower latency, higherthroughput, less user frustration, and higher QoE.

Although discussed primarily in terms of two devices communicating witheach other and the method being applied by either endpoint, in manyimplementations, the method may be applied by an intermediary devicebetween two endpoints. For example, in some implementations, a networkaccelerator may be deployed between two computing devices. The networkaccelerator may monitor bandwidth and packet loss rate for theconnections, and may insert configuration packets or modifyconfiguration settings in packets of the connections traversing thedevice, causing the endpoints to create more application layer streamswithin a connection or more transport layer connections each carryingfewer streams. Thus, the computing device discussed herein may refer toan endpoint of the transport layer connection, or an intermediary devicereceiving and forwarding packets of the connection.

Furthermore, although discussed primarily in terms of the HTTP/2 and TCPprotocols, the systems and methods discussed herein may be applied toany application layer protocol that allows or provides multiplexing ofapplication layer requests and responses within a single transport layerconnection.

FIG. 2 is a block diagram of an embodiment of a computing device 100 fordynamically optimizing the number of application layer streams that maybe multiplexed into a single transport layer connection, as well asother computing devices including client computing devices, servercomputing devices, workstation devices, cloud computing devices, or anyother type and form of computing device, referred to generally herein asa “computing device” or “computing devices”. Such devices may includelaptop computers, desktop computers, rackmount computers, tabletcomputers, wearable computers, appliances, cluster devices orappliances, server clouds or farms, virtual machines executed by one ormore physical machines, or any other type of computing device. As shownin FIG. 2, a computing device may include one or more central processingunits or processors 200, one or more network interfaces 202, one or moreinput/output controllers or devices 204, one or more memory units 206which may include system memory such as RAM as well as internal orexternal storage devices. A computing device may also include otherunits not illustrated including installation devices, display devices,keyboards, pointing devices such as a mouse, touch screen devices, orother such devices. Memory 206 may include, without limitation, anoperating system 210 and/or software.

The central processing unit 200 is any logic circuitry that responds toand processes instructions fetched from the memory 206. In manyembodiments, the central processing unit 200 is provided by amicroprocessor unit, such as: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by International BusinessMachines of White Plains, N.Y.; or those manufactured by Advanced MicroDevices of Sunnyvale, Calif. The computing device may be based on any ofthese processors, or any other processor capable of operating asdescribed herein.

Memory 206, sometimes referred to as a main memory unit, may be one ormore memory chips capable of storing data and allowing any storagelocation to be directly accessed by the microprocessor 200, such as anytype or variant of Static random access memory (SRAM), Dynamic randomaccess memory (DRAM), Ferroelectric RAM (FRAM), NAND Flash, NOR Flashand Solid State Drives (SSD). The memory 206 may be based on any of theabove described memory chips, or any other available memory chipscapable of operating as described herein. In the embodiment shown, theprocessor 200 communicates with main memory 206 via a system bus 208(described in more detail below). In other embodiments, the processorcommunicates directly with main memory 206 via a memory port. Forexample, in such embodiments, the memory 206 may be DRDRAM. In otherembodiments, processor 200 may communicate directly with cache memoryvia a secondary bus, sometimes referred to as a backside bus. In otherembodiments, the main processor 200 communicates with cache memory usingthe system bus 208. Cache memory typically has a faster response timethan memory accessible via a system bus, and is provided by, forexample, SRAM, BSRAM, or EDRAM.

In some embodiments, the processor 200 communicates with various I/Odevices 204 via local system bus 208. Various buses may be used toconnect the central processing unit 200 to any I/O devices, for example,a VESA VL bus, an ISA bus, an EISA bus, a MicroChannel Architecture(MCA) bus, a PCI bus, a PCI-X bus, a PCI-Express bus, or a NuBus. Forembodiments in which the I/O device is a video display, the processor200 may use an Advanced Graphics Port (AGP) to communicate with thedisplay. In some embodiments, the processor 200 may communicate directlywith I/O devices, for example via HYPERTRANSPORT, RAPIDIO, or INFINIBANDcommunications technology. A wide variety of I/O devices may be presentin the computing device 100. Input devices include keyboards, mice,trackpads, trackballs, microphones, dials, touch pads, touch screen, anddrawing tablets. Output devices include video displays, speakers, inkjetprinters, laser printers, projectors and dye-sublimation printers. TheI/O devices may be controlled by an I/O controller 204 as shown in FIG.2. The I/O controller may control one or more I/O devices such as akeyboard and a pointing device, e.g., a mouse or optical pen.Furthermore, an I/O device may also provide storage and/or aninstallation medium for the computing device. In still otherembodiments, the computing device may provide USB connections (notshown) to receive handheld USB storage devices such as the USB FlashDrive line of devices manufactured by Twintech Industry, Inc. of LosAlamitos, Calif.

The computing device may support any suitable installation device (notillustrated), such as a disk drive, a CD-ROM drive, a CD-R/RW drive, aDVD-ROM drive, a flash memory drive, tape drives of various formats, USBdevice, hard-drive, a network interface, or any other device suitablefor installing software and programs. The computing device may furtherinclude a storage device, such as one or more hard disk drives orredundant arrays of independent disks, for storing an operating systemand other related software, and for storing application softwareprograms such as any program or software for implementing (e.g.,configured and/or designed for) the systems and methods describedherein. Optionally, any of the installation devices could also be usedas the storage device. Additionally, the operating system and thesoftware can be run from a bootable medium.

Furthermore, the computing device may include a network interface 202 tointerface to a network through a variety of connections including, butnot limited to, standard telephone lines, LAN or WAN links (e.g.,802.11, T1, T3, 56 kb, X.25, SNA, DECNET), broadband connections (e.g.,ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET), wirelessconnections, or some combination of any or all of the above. Connectionscan be established using a variety of communication protocols (e.g.,TCP/IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, FiberDistributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 802.11a,IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, IEEE 802.11ad,CDMA, GSM, WiMax and direct asynchronous connections). In oneembodiment, the computing device communicates with other computingdevices via any type and/or form of gateway or tunneling protocol suchas Secure Socket Layer (SSL) or Transport Layer Security (TLS). Thenetwork interface 202 may include a built-in network adapter, networkinterface card, PCMCIA network card, card bus network adapter, wirelessnetwork adapter, USB network adapter, modem or any other device suitablefor interfacing the computing device to any type of network capable ofcommunication and performing the operations described herein.

In some embodiments, the computing device may include or be connected toone or more display devices. As such, any I/O devices and/or the I/Ocontroller 204 may include any type and/or form of suitable hardware,software, or combination of hardware and software to support, enable orprovide for the connection and use of the display device(s) by thecomputing device. For example, the computing device may include any typeand/or form of video adapter, video card, driver, and/or library tointerface, communicate, connect or otherwise use the display device(s).In one embodiment, a video adapter may include multiple connectors tointerface to the display device(s). In other embodiments, the computingdevice may include multiple video adapters, with each video adapterconnected to the display device(s). In some embodiments, any portion ofthe operating system 210 of the computing device may be configured forusing multiple displays. One ordinarily skilled in the art willrecognize and appreciate the various ways and embodiments that acomputing device may be configured to have one or more display devices.

In further embodiments, an I/O device may be a bridge between the systembus 208 and an external communication bus, such as a USB bus, an AppleDesktop Bus, an RS-232 serial connection, a SCSI bus, a FireWire bus, aFireWire 800 bus, an Ethernet bus, an AppleTalk bus, a Gigabit Ethernetbus, an Asynchronous Transfer Mode bus, a FibreChannel bus, a SerialAttached small computer system interface bus, a USB connection, or aHDMI bus.

A client device or server of the sort depicted in FIG. 2 may operateunder the control of an operating system 210, which control schedulingof tasks and access to system resources. The client device or server canbe running any operating system such as any of the versions of theMICROSOFT WINDOWS operating systems, the different releases of the Unixand Linux operating systems, any version of the MAC OS for Macintoshcomputers, any embedded operating system, any real-time operatingsystem, any open source operating system, any proprietary operatingsystem, any operating systems for mobile computing devices, or any otheroperating system capable of running on the computing device andperforming the operations described herein. Typical operating systemsinclude, but are not limited to: Android, produced by Google Inc.;WINDOWS 7, 8, or 10, produced by Microsoft Corporation of Redmond,Wash.; MAC OS and iOS, produced by Apple Computer of Cupertino, Calif.;WebOS, produced by Research In Motion (RIM); OS/2, produced byInternational Business Machines of Armonk, N.Y.; and Linux, afreely-available operating system distributed by Caldera Corp. of SaltLake City, Utah, or any type and/or form of a Unix operating system,among others.

As discussed above, the computer system can be any workstation,telephone, desktop computer, laptop or notebook computer, server,handheld computer, mobile telephone or other portable telecommunicationsdevice, media playing device, a gaming system, mobile computing device,or any other type and/or form of computing, telecommunications or mediadevice that is capable of communication. The computer system hassufficient processor power and memory capacity to perform the operationsdescribed herein.

In some embodiments, the computing device may have different processors,operating systems, and input devices consistent with the device. Forexample, in one embodiment, the computing device is a smart phone,mobile device, tablet or personal digital assistant. In still otherembodiments, the computing device is an Android-based mobile device, aniPhone smart phone manufactured by Apple Computer of Cupertino, Calif.,or a Blackberry or WebOS-based handheld device or smart phone, such asthe devices manufactured by Research In Motion Limited. Moreover, thecomputing device can be any workstation, desktop computer, laptop ornotebook computer, server, handheld computer, mobile telephone, anyother computer, or other form of computing or telecommunications devicethat is capable of communication and that has sufficient processor powerand memory capacity to perform the operations described herein.

A computing device 100 may execute one or more applications 208.Applications 208 may comprise any type and form of application, service,daemon, routine, or other executable logic, and may communicate withremote computing devices via application layer protocols. Applications208 may include web browsers, media players, multiplayer video games,voice over internet protocol (VoIP) or chat applications, social networkapplications, or any other type and form of applications. In someimplementations, applications 208 may include servers for communicatingwith other applications, such as web servers, file servers, streamingvideo servers, database servers, or other such applications.Applications 208 may each comprise a plurality of files, includingexecutable files, libraries, data files, parameter or preference files,registry keys, configuration files, graphics, audio files, video files,dictionaries, templates, or any other type and form of data. Anapplication 208 may exist in a plurality of versions, and may beidentified by a version number, name, or other indicator.

Computing device 100 may comprise or execute a stream manager 212.Stream manager 212 may be part of the network interface 202. Forexample, stream manager 212 may comprise circuitry such as an ASICcircuit on a network interface card. In another implementation, streammanager 212 may be executed by a coprocessor or packet processor of thenetwork interface. In still another implementation, a stream manager212′ may be part of an operating system 210, such as part of a networkdriver or system kernel. Stream manager 212 may comprise an application,service, daemon, routine, or other executable logic for monitoringbandwidth and packet loss of a transport layer connection, and foradjusting the value of a concurrent stream parameter for applicationlayer requests and responses multiplexed via the connection. In someimplementations, stream manager 212 may be configured to performadditional processing for the connection, such as generatingconfiguration or settings packets for transmission by the networkinterface, modifying parameters of a network stack, etc.

FIG. 3 is a flowchart of an embodiment of a method for dynamicallyoptimizing the number of application layer streams that may bemultiplexed into a single transport layer connection. At step 300, adevice may establish a communication session with a second device. Thecommunication session may comprise a plurality of concurrent applicationlayer streams carried via a transport layer connection, such as aplurality of multiplexed HTTP/2 packets or payloads encapsulated in asingle transport layer header. In some implementations, establishing thecommunication session may comprise performing a handshaking procedure,establishing parameters for the communication, or other such functions.As discussed above, the device may be an endpoint of the communication(e.g. a client, server, or other end point device), or may be anintermediary device receiving and forwarding communications to other endpoint devices. Thus, the transport layer connection may or may notterminate at the device.

At step 302, the device may identify a net bandwidth of the transportlayer connection. Identifying the bandwidth may comprise measuring athroughput of the connection, a window size of the connection, a roundtrip time of the connection, or other such parameters representative ofdata throughput on the connection. For example, the device may measurethe time to receive an acknowledgement to a transmission to the otherdevice. In some implementations, measuring the bandwidth may comprisedividing a window size of the connection by a round-trip time for theconnection.

At step 304, the device may determine if the net bandwidth is greaterthan a predetermined threshold. The threshold may be set by anadministrator, manufacturer, or user, or may be dynamically determined,e.g., based on overall throughput of the device. For example, in someimplementations, the threshold may be set to a fraction of a totalthroughput of the device.

If the net bandwidth is not greater than the predetermined threshold,then in some implementations, at step 306, the device may set a streamlimit to a first predetermined value. The first predetermined value maybe a default value, or may be a value lower than a predetermined defaultvalue. The value may be, for example, 10 concurrent application layerstreams, 5 streams, 15 streams, or any other such value.

If the net bandwidth is greater than the predetermined threshold, thenat step 308, the device may determine a packet loss rate for theconnection. In some implementations, step 308 may be performed prior tostep 304 and/or step 302. In some implementations, the packet loss ratemay be measured by identifying a number of dropped packets or notreceived packets within a predetermined period, and/or by measuring anumber of unacknowledged transmissions or retransmissions within thepredetermined period. In some implementations, the device may receive apacket loss rate measurement from the remote device.

At step 310, the device may determine if the packet loss rate is greaterthan a second predetermined threshold. The threshold may be set by anadministrator, manufacturer, or user, or may be dynamically determined,e.g., based on a fraction of throughput of the device. The secondpredetermined threshold may be different from the first predeterminedthreshold or net bandwidth threshold and use different units (e.g.dropped or lost packets per second, compared to bytes, kilobytes, ormegabytes per second).

If the packet loss rate is not above the second threshold, then at step312, the device may set a stream limit to a second predetermined valuethat may be greater than the first predetermined value of step 306. Forexample, the value may be 15 concurrent application layer streams, 10streams, 20 streams, or any other such value, referred to generally as a“high” value in comparison to the “medium” value of step 306. The highvalue may be utilized because, if the packet loss rate is low, thedevice may not need to take congestion avoidance mitigation steps, asdiscussed above.

If the packet loss rate is above the second threshold, then at step 314,the device may set a stream limit to a third predetermined value thatmay be less than the first predetermined value of step 306 (andtherefore less than the second predetermined value of step 312). Forexample, the value may 5 concurrent application layer streams, 2streams, 10 streams, or any other such value, referred to generally as a“low” value in comparison to the medium value of step 306. The low valuemay be utilized because, if the packet loss rate is high (but the netbandwidth for the connection is above the first threshold), the systemmay mitigate congestion avoidance by spawning additional transport layerconnections to carry application layer requests instead of multiplexingthem into the established transport layer connection.

At step 316, in some implementations, the device may determine if apresent number of concurrent streams is greater than the stream limitset at steps 306, 312, or 314. If not, then processing may continue andsteps 302-316 may be repeated. Conversely, if the number of streams isgreater than the limit, at step 318, the device may transmit a streamlimit command to the other device. Transmitting the limit command maycomprise transmitting a settings or configuration packet having aconcurrent stream limit value equal to the stream limit, such as apacket comprising a SETTINGS_MAX_CONCURRENT_STREAMS value equal to thestream limit. The limit command may be sent on its own, or may beprovided as part of a header (e.g. application layer header or transportlayer header) of a packet of the established connection. Receipt of thelimit command may cause the remote device, if it has further applicationlayer data to send, to establish a second transport layer connection(e.g. initiating a new iteration of the method, starting at step 300).

In some implementations, the limit command 318 may be sent as part ofevery packet, and step 316 may be skipped. For example, in someimplementations, the concurrent stream limit value may be sent in theheader of every application layer packet, and thus, no determination ofwhether to send the value is required. In other implementations, at step316 or in a separate step (not illustrated), the device may determine ifthe present number of streams is less than the limit, or less than thelimit minus a predetermined value (e.g. less than the limit minus one,or the limit minus five, or any other such value). If so, the device maytransmit the limit commend at step 318. This may allow the device toincrease the number concurrent streams, for example after congestion orpacket losses have recovered.

Thus, the systems and methods discussed herein provide for dynamicallyoptimizing the number of application layer streams that may bemultiplexed into a single transport layer connection, providing theadvantages of application layer multiplexing without incurringunnecessary congestion-based network delays. The device may dynamicallyincrease and decrease a number of concurrent application layer streamsto balance throughput and congestion avoidance.

Although the disclosure may reference one or more “users”, such “users”may refer to user-associated devices or stations (STAs), for example,consistent with the terms “user” and “multi-user” typically used in thecontext of a multi-user multiple-input and multiple-output (MU-MIMO)environment.

Although examples of communications systems described above may includedevices and APs operating according to an 802.11 standard, it should beunderstood that embodiments of the systems and methods described canoperate according to other standards and use wireless communicationsdevices other than devices configured as devices and APs. For example,multiple-unit communication interfaces associated with cellularnetworks, satellite communications, vehicle communication networks, andother non-802.11 wireless networks can utilize the systems and methodsdescribed herein to achieve improved overall capacity and/or linkquality without departing from the scope of the systems and methodsdescribed herein.

It should be noted that certain passages of this disclosure mayreference terms such as “first” and “second” in connection with devices,mode of operation, transmit chains, antennas, etc., for purposes ofidentifying or differentiating one from another or from others. Theseterms are not intended to merely relate entities (e.g., a first deviceand a second device) temporally or according to a sequence, although insome cases, these entities may include such a relationship. Nor do theseterms limit the number of possible entities (e.g., devices) that mayoperate within a system or environment.

It should be understood that the systems described above may providemultiple ones of any or each of those components and these componentsmay be provided on either a standalone machine or, in some embodiments,on multiple machines in a distributed system. In addition, the systemsand methods described above may be provided as one or morecomputer-readable programs or executable instructions embodied on or inone or more articles of manufacture. The article of manufacture may be ahard disk, a CD-ROM, a flash memory card, a PROM, a RAM, a ROM, or amagnetic tape. In general, the computer-readable programs may beimplemented in any programming language, such as LISP, PERL, C, C++, C#,PROLOG, or in any byte code language such as JAVA. The software programsor executable instructions may be stored on or in one or more articlesof manufacture as object code.

While the foregoing written description of the methods and systemsenables one of ordinary skill to make and use what is consideredpresently to be the best mode thereof, those of ordinary skill willunderstand and appreciate the existence of variations, combinations, andequivalents of the specific embodiment, method, and examples herein. Thepresent methods and systems should therefore not be limited by the abovedescribed embodiments, methods, and examples, but by all embodiments andmethods within the scope and spirit of the disclosure.

It should be understood that the systems described above may providemultiple ones of any or each of those components and these componentsmay be provided on either a standalone machine or, in some embodiments,on multiple machines in a distributed system. The systems and methodsdescribed above may be implemented as a method, apparatus or article ofmanufacture using programming and/or engineering techniques to producesoftware, firmware, hardware, or any combination thereof. In addition,the systems and methods described above may be provided as one or morecomputer-readable programs embodied on or in one or more articles ofmanufacture. The term “article of manufacture” as used herein isintended to encompass code or logic accessible from and embedded in oneor more computer-readable devices, firmware, programmable logic, memorydevices (e.g., EEPROMs, ROMs, PROMs, RAMs, SRAMs, etc.), hardware (e.g.,integrated circuit chip, Field Programmable Gate Array (FPGA),Application Specific Integrated Circuit (ASIC), etc.), electronicdevices, a computer readable non-volatile storage unit (e.g., CD-ROM,hard disk drive, etc.). The article of manufacture may be accessiblefrom a file server providing access to the computer-readable programsvia a network transmission line, wireless transmission media, signalspropagating through space, radio waves, infrared signals, etc. Thearticle of manufacture may be a flash memory card or a magnetic tape.The article of manufacture includes hardware logic as well as softwareor programmable code embedded in a computer readable medium that isexecuted by a processor. In general, the computer-readable programs maybe implemented in any programming language, such as LISP, PERL, C, C++,C#, PROLOG, or in any byte code language such as JAVA. The softwareprograms may be stored on or in one or more articles of manufacture asobject code.

While various embodiments of the methods and systems have beendescribed, these embodiments are illustrative and in no way limit thescope of the described methods or systems. Those having skill in therelevant art can effect changes to form and details of the describedmethods and systems without departing from the broadest scope of thedescribed methods and systems. Thus, the scope of the methods andsystems described herein should not be limited by any of theillustrative embodiments and should be defined in accordance with theaccompanying claims and their equivalents.

What is claimed is:
 1. A method for managing concurrent applicationlayer streams, comprising: establishing, by a network interface of afirst device, a communication session with a second device, thecommunication session comprising a plurality of concurrent applicationlayer streams; determining, by a stream manager of the first device,that a net bandwidth utilization of the communication session is above afirst predetermined threshold; determining, by the stream managerresponsive to determining that the net bandwidth utilization of thecommunication session is above the first predetermined threshold, that apacket loss rate of the communication session is above a secondpredetermined threshold; and transmitting, by the network interface tothe second device, a first request to reduce a number of the pluralityof concurrent streams, responsive to the determination that the packetloss rate of the communication session is above the second predeterminedthreshold.
 2. The method of claim 1, wherein transmitting the firstrequest to reduce the number of the plurality of concurrent streamscomprises transmitting a packet comprising aSETTINGS_MAX_CONCURRENT_STREAMS parameter with a predetermined valuesmaller than the number of the plurality of concurrent streams.
 3. Themethod of claim 1, further comprising subsequently determining, by thestream manager, that the packet loss rate of the communication sessionis below the second predetermined threshold; and transmitting, by thenetwork interface to the second device, a second request to increase thenumber of the plurality of concurrent streams, responsive to thedetermination that the packet loss rate of the communication session isbelow the second predetermined threshold.
 4. The method of claim 3,further comprising subsequently determining, by the stream manager, thatthe net bandwidth utilization of the communication session is below thefirst predetermined threshold; and responsive to determining that thenet bandwidth utilization of the communication session is below thefirst predetermined threshold, transmitting, by the network interface tothe second device, a second request to reduce the number of theplurality of concurrent streams.
 5. The method of claim 4, wherein thefirst request comprises an identification of a lower number ofconcurrent streams than the second request.
 6. The method of claim 1,wherein the communication session comprises a hypertext transportprotocol v2 (HTTP2) session.
 7. The method of claim 1, whereintransmitting the first request to reduce the number of the plurality ofconcurrent streams comprises transmitting a settings frame comprising anidentification of one or more configuration parameters for thecommunications session.
 8. A method for managing concurrent applicationlayer streams, comprising: establishing, by a network interface of afirst device, a communication session with a second device, thecommunication session comprising a plurality of concurrent applicationlayer streams; receiving, by the network interface from the seconddevice, a first request to reduce a number of the plurality ofconcurrent streams, transmitted responsive to a determination by thesecond device that (i) a net bandwidth utilization of the communicationsession is above a first predetermined threshold, and (ii) a packet lossrate of the communication session is above a second predeterminedthreshold; and transmitting via a first stream of the plurality ofconcurrent streams, by the network interface responsive to receipt ofthe first request to reduce the number of the plurality of concurrentstreams, a packet comprising an indicator that at least one stream ofthe plurality of concurrent application layer streams is closed.
 9. Themethod of claim 8, wherein receiving the first request to reduce thenumber of the plurality of concurrent streams comprises receiving apacket comprising a SETTINGS_MAX_CONCURRENT_STREAMS parameter with apredetermined value smaller than the number of the plurality ofconcurrent streams.
 10. The method of claim 8, further comprisingreceiving a second request to increase the number of the plurality ofconcurrent streams, by the first device from the second device,transmitted responsive to a determination by the second device that thepacket loss rate of the communication session is below the secondpredetermined threshold; and transmitting via the first stream of theplurality of concurrent streams, by the network interface responsive toreceipt of the second request to increase the number of the plurality ofconcurrent streams, a second packet comprising the plurality ofconcurrent application layer streams.
 11. The method of claim 10,further comprising receiving a second request to reduce the number ofthe plurality of concurrent streams, by the first device from the seconddevice, transmitted responsive to a determination by the second devicethat the net bandwidth utilization of the communication session is belowthe first predetermined threshold; and transmitting via the first streamof the plurality of concurrent streams, by the network interfaceresponsive to receipt of the second request to reduce the number of theplurality of concurrent streams, a packet comprising an indicator that asecond at least one stream of the plurality of concurrent applicationlayer streams is closed.
 12. The method of claim 11, wherein the firstrequest comprises an identification of a lower number of concurrentstreams than the second request.
 13. The method of claim 8, wherein thecommunication session comprises a hypertext transport protocol v2(HTTP2) session.
 14. The method of claim 8, wherein receiving the firstrequest to reduce the number of the plurality of concurrent streamscomprises receiving a settings frame comprising an identification of oneor more configuration parameters for the communications session.
 15. Asystem for managing concurrent application layer streams, comprising: anetwork interface configured to establish a communication session with asecond device, the communication session comprising a plurality ofconcurrent application layer streams; and a stream manager, configuredto: determine that a net bandwidth utilization of the communicationsession is above a first predetermined threshold, and determine,responsive to determining that the net bandwidth utilization of thecommunication session is above the first predetermined threshold, that apacket loss rate of the communication session is above a secondpredetermined threshold; and wherein the network interface is furtherconfigured to transmit, to the second device, a first request to reducea number of the plurality of concurrent streams, responsive to thedetermination that the packet loss rate of the communication session isabove the second predetermined threshold.
 16. The system of claim 15,wherein the network interface is further configured to transmit a packetcomprising a SETTINGS_MAX_CONCURRENT_STREAMS parameter with apredetermined value smaller than the number of the plurality ofconcurrent streams.
 17. The system of claim 15, wherein the streammanager is further configured to subsequently determine that the packetloss rate of the communication session is below the second predeterminedthreshold; and wherein the network interface is further configured totransmit, to the second device, a second request to increase the numberof the plurality of concurrent streams, responsive to the determinationthat the packet loss rate of the communication session is below thesecond predetermined threshold.
 18. The system of claim 17, wherein thestream manager is further configured to subsequently determine that thenet bandwidth utilization of the communication session is below thefirst predetermined threshold; and wherein the network interface isfurther configured to transmit, to the second device, a second requestto reduce the number of the plurality of concurrent streams, responsiveto determining that the net bandwidth utilization of the communicationsession is below the first predetermined threshold.
 19. The system ofclaim 18, wherein the first request comprises an identification of alower number of concurrent streams than the second request.
 20. Thesystem of claim 15, wherein the network interface is further configuredto transmit a settings frame comprising an identification of one or moreconfiguration parameters for the communications session.